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Abstract — This work considers the problem of transmitting 
multiple compressible sources over a network at minimum cost. 
The aim is to find the optimal rates at which the sources should 
be compressed and the network Hows using which they should 
be transmitted so that the cost of the transmission is minimal. 
We consider networks with capacity constraints and linear cost 
functions. The problem is complicated by the fact that the 
description of the feasible rate region of distributed source coding 
problems typically has a number of constraints that is exponential 
in the number of sources. This renders general purpose solvers 
inefficient. We present a framework in which these problems can 
be solved efficiently by exploiting the structure of the feasible 
rate regions coupled with dual decomposition and optimization 
techniques such as the subgradient method and the proximal 
bundle method. 

Index Terms — minimum cost network flow, distributed source 
coding, network coding, convex optimization, dual decomposition. 



I. Introduction 

In recent years the emergence of sensor networks [1] as a 
new paradigm has introduced a number of issues that did not 
exist earlier. Sensor networks have been considered among 
other things by the military for battlefields, by ecologists 
for habitat monitoring and even for extreme event warning 
systems. These networks consist of tiny, low-power nodes 
that are typically energy constrained. In general, they also 
have low-computing power Thus, designing efficient sensor 
networks requires us to address engineering challenges that are 
significantly different from the ones encountered in networks 
such as the Internet. One unique characteristic of sensor net- 
works is that the data that is sensed by different sensor nodes 
and relayed to a terminal is typically highly correlated. As an 
example consider a sensor network deployed to monitor the 
temperature or humidity levels in a forest. The temperature is 
not expected to vary significantly over a small area. Therefore 
we do expect that the readings corresponding to nearby sensors 
are quite correlated. It is well-known that the energy consumed 
in transmission by a sensor is quite substantial and therefore 
efficient low power methods to transfer the data across the 
network are of interest. This leads us to investigate efficient 
techniques for exploiting the correlation of the data while 
transmitting it across the network. There are multiple ways 
in which the correlation can be exploited, 
a) The sensor nodes can communicate amongst themselves 
to inform each other of the similarity of their data 
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and then transmit only as much data as is required. 
This comes at the cost of the overhead of inter-sensor 
communication, 
b) The sensors can choose to act independently and still 
attempt to transmit the compressed data. This strategy is 
likely to be more complicated from the receiver's point 
of view. 

Usually the terminal to which the data is transmitted has 
significantly more resources (energy, computing power etc.). 
Thus, the latter solution is more attractive from a network 
resource efficiency point of view. The question of whether 
the distributed compression of correlated sources can be as 
efficient as their compression when the sources communicate 
with each other was first considered and answered in the 
affirmative by Slepian and Wolf in their famous paper [2]. 
A number of authors [3] [4] have investigated the construction 
of coding techniques that achieve the Slepian- Wolf bounds and 
also proposed their usage in sensor networks [5]. 

New paradigms have also emerged recently in the area 
of network information transfer Traditionally information 
transfer over networks has been considered via routing. Data 
packets from a source node are allowed to be replicated 
and forwarded by the intermediate nodes so that terminal 
nodes can satisfy their demands. However, network coding 
offers an interesting alternative where intermediate nodes in 
a network have the ability to forward functions of incoming 
packets rather than copies of the packets. The seminal work of 
Ahlswede et al. [6] showed that network coding achieves the 
capacity of single-source multiple-terminal multicast where 
all the terminals are interested in receiving the same set of 
messages from the source. This was followed by a number of 
works that presented constructions and bounds for multicast 
network codes [7] [8]. More recently, there has been work 
[9] [10] on characterizing rate regions for arbitrary network 
connections where the demands of the terminals can be 
arbitrary. 

Given these developments in two different fields, a natural 
question to ask is how can one transmit compressible sources 
over a network using network coding and whether this can be 
done efficiently. This problem was considered by Song and 
Yeung [11] and Ho et al. [12]. They showed that as long as 
the minimum cuts between all nonempty subsets of sources 
and a particular terminal were sufficiently large, random linear 
network coding over the network followed by appropriate 
decoding at the terminals achieves the Slepian- Wolf bounds. 
The work of Ramamoorthy et al. [13] investigated the perfor- 
mance of separate source and network codes and showed that 
separation does not hold in general. Both these papers only 



considered capacity constraints on the edges of the network 
and did not impose any cost associated with edge usage. 

In the networking context the problem of minimum cost 
network flow has been widely investigated. Here, every edge in 
the network has a cost per unit flow associated with it. The cost 
of a given routing solution is the sum of the costs incurred over 
all the hnks. The problem is one of finding network flows such 
that the demand of the terminals is satisfied at minimum cost. 
This problem has been very well investigated in the routing 
context [14]. The problem of minimum cost multicast using 
network coding was considered by Lun et al. [15] and they 
presented centralized and distributed solutions to it. 

In this paper we consider the problem of minimum cost 
joint rate and flow allocation over a network that is utilized 
for communicating compressible sources. We consider the sce- 
nario when the compression is to be performed in a distributed 
manner. The sources are not allowed to communicate with 
each other The main issue with joint rate and flow allocation 
is that typically the feasible rate region for the recovery of 
the sources (e.g. the Slepian-Wolf region) is described by 
a number of inequalities that is exponential in the number 
of sources. Thus, using a regular LP solver for solving the 
corresponding linear programming problem will be inefficient. 
In our work, we only consider networks where the links are 
independent and where transmission up to the link's capacity 
is assumed to be error free. In general, the capacity region 
characterization of more complex networks such as wireless 
networks will need to take account issues such as interference. 
Moreover, it would introduce related issues such as scheduling. 
We do not consider these problems in this work. 

A. Main Contributions 

The main contributions of this paper are as follows. We 
present a framework in which minimum cost problems that 
involve transmitting compressible sources over a network in 
a distributed manner can be solved efficiently. We consider 
general linear cost functions, allow capacity constraints on 
the edges of the network and consider the usage of network 
coding. The following problems are considered. 

a) Slepian-Wolf over a network. The sources are assumed to 
be discrete and memoryless and they need to be recovered 
losslessly [2] at the terminals of the network. We address 
the problem of jointly finding the operating rate vectors 
for the sources and the corresponding network flows that 
allow lossless recovery at the terminals at minimum cost. 

b) Quadratic Gaussian CEO over a network. A Gaussian 
source is observed by many noisy sensors and needs to be 
recovered at the terminal subject to a quadratic distortion 
constraint [16]. We present a solution to the problem of 
joint rate and network flow allocation that allows recovery 
at the terminal at minimum cost. 

c) Lifetime maximization of sensor networks with distortion 
constraints. A Gaussian source observed by many noisy 
sensors needs to be recovered at the terminal with a 
certain fidelity. We are interested in finding routing flows 
that would maximize the lifetime of the network. 

We demonstrate that these problems can be solved efficiently 
by exploiting the structure of the feasible rate regions coupled 



with dual decomposition techniques and subgradient methods 
[17]. 

B. Related Work 

Problems of a similar flavor have been examined in several 
papers. Cristescu et al. considered the Networked Slepian-Wolf 
problem [18] and the case of lossy correlated data gathering 
over a network [19], but did not impose capacity constraints 
on the edges. Their solutions only considered very specific 
types of cost functions. The work of Li & Ramamoorthy [20], 
[21] and Roumy & Gesbert [22] considered a rate allocation 
under pairwise constraints on the distributed source code 
used for compression. The work of Liu et al. [23] and [24] 
considers a related problem, where they seek to minimize the 
total communication cost of a wireless sensor network with 
a single sink. They show that when the link communication 
costs are convex, then the usage of Slepian-Wolf coding and 
commodity flow routing is optimal. Moreover, they introduce 
the notion of distance entropy, and show that under certain 
situations the distance entropy is the minimum cost achieved 
by Slepian-Wolf coding and shortest path routing. They also 
propose hierarchical transmission schemes that exploit corre- 
lation among neighboring sensor nodes, and do not require 
global knowledge of the correlation structure. These schemes 
are shown to be order-optimal in specific situations. The 
main difference between our work and theirs, is the fact 
that we consider network coding and networks with multiple 
terminals. Moreover, in the case of general convex link cost 
functions, their focus is on showing that Slepian-Wolf coding 
and commodity flow routing is optimal. They do not consider 
the problem of actually finding the optimal flows and rates. 

A problem formulation similar to ours was introduced by 
Barros et al. [25] but they did not present an efficient solution 
to it. The problem of exponentially many constraints has been 
noted by other authors as well [26] [27]. 

The approach in our work is inspired by the work of Yu 
et al. [28]. However since our cost functions only penalize 
the usage of links in the network, we are effectively able 
to exploit the structure of the feasible rate region to make 
our overall solution efficient. In addition we explicitly derive 
the dual function and the corresponding update equations for 
maximizing it based on the specific structure of the rate region. 
Furthermore, we consider applications in network coding and 
lifetime maximization in sensor networks that have not been 
considered previously. In concurrent and independent work 
[29] presented some approaches similar to ours (see also [30], 
where the case of two sources is discussed). However our 
approach has been applied to the minimum cost quadratic 
Gaussian CEO problem over a network and lifetime maxi- 
mization with distortion constraints that were not considered 
in [29]. 

A reviewer has pointed out that the problem of gener- 
alizing the Slepian-Wolf theorem to the network case was 
first considered by Han [31] in 1980. However, in [31] only 
networks with a single terminal were considered. In the single 
terminal case the corresponding flows can be supported by 
pure routing. Interestingly, in the same paper, Han references 
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the work of Fujishige [32] that studies the optimal independent 
flow problem (this was also pointed by the same reviewer). 
Fujishige's work considers a network flow problem that has 
polymatroidal [33] constraints for the source values and the 
terminal values. In particular, if there is only one terminal, then 
this algorithm provides an efficient solution to the minimum 
cost Slepian-Wolf problem over a network. However, it is 
unclear whether it can be extended to the case of multiple 
terminals and network coding. We discuss Fujishige's work in 
more detail in Section |III1 after the precise problem has been 
formulated. 

This paper is organized as follows. Section HI] overviews 
the notation and the broad setup under consideration in this 
paper Section [III] formulates and solves the minimum cost 
Slepian-Wolf problem over a network, Section|IV]discusses the 
quadratic Gaussian CEO problem over a network and Section 
rvl presents and solves the problem of lifetime maximization 
of sensor networks when the source needs to be recovered 
under distortion constraints. Each of these sections also present 
simulation results that demonstrate the effectiveness of our 
method. Section|Vl]discusses the conclusions and future work. 

II. Preliminaries 

In this section we introduce the basic problem setup that 
shall be used in the rest of this paper In subsequent sections 
we shall be presenting efficient algorithms for solving three 
different problems that fall under the umbrella of distributed 
source coding problems over a network. We shall present 
the exact formulation of the specific problem within those 
sections. We are given the following. 

a) A directed acyclic graph G = (V, C) that represents 
the network. Here V represents the set of vertices, E the 
set of edges and Cij,{i,j) G E is the capacity of the 
edge in bits/transmission. The edges are assumed 
to be error-free and the capacity of the edges is assumed 
to be rational. We are also given a set of source nodes 
S C V where 151 = Ns and a set of terminal nodes 
T C V where |T| = Nj^. Without loss of generality we 
assume that the vertices are numbered so that the vertices 
1,2,..., Ns correspond to the source nodes. 

b) A set of sources Xi,X2, ■ ■ ■ X^^, such that the i*'' source 
is observed at source node i G S. The values of the 
sources are drawn from some joint distribution and can 
be either continuous or discrete. 

Based on these we can define the capacity region of the 
terminal Tj £ T with respect to S as 

Ct, = {(i?i,...,i?jvs) :VBC5,^i?, <min-cut(B,T,)}. 

zG-B 

Thus, Ct consists of inequalities that define the maximum 
flow (or minimum cut) from each subset of S to the terminal 
Tj. A rate vector {Ri, . . . , Rn^) e Ct can be transmitted 
from the source nodes 1, . . . , Ns to terminal Tj via routing 
[14]. In the subsequent sections we shall consider different 
minimum cost problems involving the transmission of the 
sources over the network to the terminals. 



III. Minimum cost Slepian-Wolf over a network 

Under this model, the sources are discrete and memoryless 
and their values are drawn i.i.d. from a joint distribution 
p{xi, . . . , XNg). The i*'' source node only observes Xi for 
i E S. The different source nodes operate independently and 
are not allowed to communicate. The source nodes want to 
transmit enough information using the network to the terminals 
so that they can recover the original sources, losslessly. 

This problem was first investigated in the seminal paper of 
Slepian and Wolf [2] where they considered the sources to be 
connected to the terminal by a direct link and the links did not 
have capacity constraints. The celebrated result of [2] states 
that the independent source coding of the sources Xi,i = 
1, . . . , Ns can be as efficient as joint coding when the sources 
need to be recovered error-free at the terminal. 

Suppose that for the classical Slepian-Wolf problem, the 
rate of the i*'* source is Ri. Let Xb denote the vector of 
sources {X^^, Xi^, . . . , X^^^^), for ik <E B,k = 1,...,|B|. 
The feasible rate region for this problem is given by 

7^5W = {{Ri, . . . , Rns) ■■yBCS,Y, R^ > H{Xb\Xb^)} 

The work of Csiszar [34] showed that linear codes are suffi- 
cient to approach the Slepian-Wolf (henceforth S-W) bounds 
arbitrarily closely. 

Note that the original S-W problem does not consider the 
sources to be communicating with the terminal (or more 
generally multiple terminals) over a network. Furthermore, 
there are no capacity constraints on the edges connecting the 
sources and the terminal. In situations such as sensor networks, 
where the sensor nodes are typically energy-constrained, we 
would expect the source nodes to be in communication with 
the terminal node over a network that is both capacity and 
cost-limited. Capacity constraints may be relatively strict since 
a significant amount of power is consumed in transmissions. 
The costs of using different links could be used to ensure that 
a certain part of the network is not overused resulting in non- 
uniform depletion of resources. Thus the problem of transmit- 
ting correlated data over a network with capacity constraints 
at minimum cost is of interest. We define an instance of the 
S-W problem over a network by P =< TZsw, G, S, T >. 

The transmission schemes based on linear codes (such as 
those in [34]) are based on block-wise coding, i.e., each 
source encodes N source symbols at a time. An edge with 
capacity Cij bits/transmission can transmit [NGijl bits per 
block. Conceptually, each edge can be regarded as multiple 
unit capacity edges, with each unit capacity edge capable of 
transmitting one bit per block. When communicating a block 
of length N, we consider the graph = {V, E, [NG\ ), or 
equivalently the graph {V,En, 1) (where 1 denotes a vector 
of ones) where En splits each edge from E into unit capacity 
edges. 

To facilitate the problem formulation we construct an aug- 
mented graph G* where we append a virtual super source node 
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s* to G, so that 

V* = 
E* = 

CZ = 



X, 



5Z ■"y 

{il(^j)e-B-} 



X 



{il(i:06S*} 



An) 



ViJ{s*], 

{{s*,v)\ V e S]\JE, and 

if i = s* and j e S*. 



fori e V\Tk e T, 
a;lT^^>i?p-\ for 5, TfeGT 
R^^"^ G 7^5W, for Tfc G T 



where 



We let G* = {V*,E*,C*). 

Definition 1: Feasibility. Consider an instance of the S-W 
problem over a network, P ~< TZsw, G, S,T >. Let be 
the capacity region of each receiver Ti E T with respect to S. 
If 

7^5wnCT. ^0,Vz = i,...,7Vij, 

then the feasibility condition is said to be satisfied and P is 
said to be feasible. 

The next theorem (from [12]) implies that as long as the 
feasibility condition is satisfied, random linear network coding 
over G^ followed by appropriate decoding at Ti suffices to 
reconstruct the sources Xi, X2, ■ ■ ■ , error-free at Ti. 

Theorem 1: Sufficiency of the feasibility condition [12]. 
Consider an instance of the S-W problem over a network, 
P =< TZsw, G, S,T >. If the feasibility condition (Definition 
[U is satisfied, then random linear network coding over G^ 
followed by minimum-entropy [34] or maximum-likelihood 
decoding can recover the sources at each terminal in T with 
the probability of decoding error going to as iV cx). 
The proof of the necessity of the feasibility condition can be 
found in [31]. 

It follows that if Gt, n TZsw 7^ (f> for all T, G T, it is 
sufficient to perform random linear network coding over a 
subgraph of G where the feasibility condition continues to 
be satisfied. The question then becomes, how do we choose 
appropriate subgraphs? 

For this purpose, we now present the formulation of the 
minimum cost S-W problem over a network. 

Let x\J'''^ represent the flow variable for edge over G* 
corresponding to the terminal Tk for G T and Zij represent 
the max-of-flows variable, maxr^gr ^ij''^ for edge (i, j). Note 
that under network coding the physical flow on edge (i,j) will 
be Zij. The variable x-^'' , represents the virtual flow variable 
over edge for terminal Tk [15]. 

We introduce variables Rl * , z = 1, . . . , Ns that represent 
the operating S-W rate variables for each terminal. Thus 




if i = s* 
if^^Tk 
otherwise 



(2) 



(3) 
(4) 



(5) 



The constraints in dU, ^ and ^ are precisely the for- 
mulation of the minimum cost single-source multiple ter- 
minal multicast with network coding for a total rate of 
H{Xi, X2, ■ . . , Xn^). The constraint (O enforces the flow 
^ (corresponding to terminal Tk from s* through source i) 
to be at least R^^*"^ . Constraint (|4]i ensures that the rate vectors 
R^'^^^ belong to the Slepian-Wolf region TZgw. A proof that 
the total rate can be fixed to be exactly H{Xi,X2, . . . ,Xn^) 
for each terminal can be found in Appendix-I. 

Suppose there exists a feasible solution z, (a;^^''^ 7?^^*=^) for 
Tk G T to MIN-COST-SW-NETWORK. Let 



E*' 



{(i,j) G E*\ z,j > 0}, and 







if {i,j)eE*^ 

otherwise. 



An) 



for terminal Tk. Let > 0,{i,j) G E, f,j = 0,(i,j) G 
E* \E represent the cost for transmitting at a unit flow over 
edge We are interested in the following optimization 

problem that we call MIN-COST-SW-NETWORK. 



minimize Zij 

s. t. < ^ < z,, < G*j, 



We define the subgraph of G* induced by z to be the 
graph G* = {V*^ , E*^ ,G*^) and the corresponding graph 
over block length N as G*^ = (V*'' , E*^' ,1). The sub- 
graphs induced by x'^^''^ can be defined analogously. We now 
show that if MIN-COST-SW-NETWORK is feasible then the 
subgraph induced by the feasible z continues to satisfy the 
condition in definition [T] and therefore it suffices to perform 
random linear network coding over this subgraph followed by 
appropriate decoding at the terminals to recover the sources. 

Lemma 1: Suppose that there exists a feasible solution z, 
(a;(^'=),i?('^'=)) for G T to MIN-COST-SW-NETWORK. 
Then, random linear network coding over the subgraph G*^ 
induced by z followed by maximum likelihood decoding at the 
terminals can recover the sources Xi,i G S at each terminal 
in r as TV ^ 00. 

Proof: To simplify the presentation we assume that all 
Cij,{i,j) G E* and H{Xb/Xb'-),B C 5 are rational 
and the block length N is large enough so that NGij and 
NH{Xb/Xb') are integral. For each terminal Tk we shall 
show that min-cut(B, Tfe) > NH{Xb\Xb<') over G*^ and 
then use Theorem [1] 

Consider a terminal Ti G T. We are given the existence 
of a feasible solution {z,x,R) from which we can find the 
corresponding flow for Ti denoted by x^'^^ \ Now consider the 
subgraph of G*^ induced by x^'^^\ Since x'^^-* is feasible, 
it supports a rate of NH{Xi, X2, . . . ,Xns) from s* to Ti 
which implies (using Menger's theorem [35]) that there exist 
NH{Xi, X2, . . . , Xns) edge-disjoint paths from s* to Ti. 
(i, j) G E* ,Tk G T (1) Furthermore at least NR^^^^ of those edge-disjoint paths 



) represents the rate vector 
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connect source node i (where i E 5) to Ti. It follows that 
if B C S then the number of edge disjoint paths from B to 
Ti is greater than or equal to -/VX^ies -^l^^''- 

Now, note that e 'R-sw which implies that for all 

BCS 

nY^R^'^ > NH{Xb\Xb^). 

This means that there exist at least NH{Xb\Xb'') edge- 
disjoint paths from B to Ti in the subgraph induced by Ti 
which in turn implies that min-cut(B, Ti) > NHiXBlXB") 
over the subgraph induced by x^^^-*. This holds for all Tk € T, 
as we have a feasible a;'^*"^ for all the terminals. Finally 
z induces a subgraph where this property continues to hold 
true for each terminal since x^J'''' < Zij, for all G 
E*,Tk G T. Therefore for each terminal Tk we have shown 
that min-cut(B,rfe) > NH{Xb\Xb'') over G*^ for all 
sufficiently large N . Using Theorem [T] we have the required 
proof. ■ 
The formulation of MIN-COST-SW-NETWORK as presented 
above is a linear program and can potentially be solved by a 
regular LP solver However the number of constraints due to 
the requirement that R G TZsw is |r|(2^^' — 1) that grows ex- 
ponentially with the number of sources. For regular LP solvers 
the time complexity scales with the number of constraints 
and variables. Thus, using a regular LP solver is certainly not 
time-efficient. Moreover even storing the constraints consumes 
exponential space and thus using a regular LP solver would 
also be space-inefficient. In the sequel we present efficient 
techniques for solving this problem. 

A. Solving MIN-COST-SW-NETWORK via dual decomposi- 
tion 

Suppose that we are given an instance of the S-W problem 
over a network specified by P =< Rsw,G,S,T >. We 
assume that P is feasible. The MIN-COSTSW-NETWORK op- 
timization problem is a linear program and therefore feasibility 
implies that strong duality holds [36]. 

We shall refer to the variables z, x^'^''\ R^'^>''> for T^ G T 
as the primal variables. To simplify notation we let xi^P^ = 
[x'l^'l^ 2^ 1^2'' • ■ • ^^s'Ns^'^ denote the vector of flow variables 
corresponding to terminal Tk on the edges from the virtual 
super node s* to the source nodes in S. We form the 
Lagrangian of the optimization problem with respect to the 
constraints 7?^^"' < x^fj^ for i e S,Tk € T. This is given 
by 

L(A, z, xC^i), . . . , a;(^"H\ . . . , E^^^h^) 

fe=i 

where A = [Xj X2 ■ ■ ■ A^^]^ is the dual variable such that 
A ^ (where ^ denotes component-wise inequality). 

For a given A, let g{X) denote the dual function obtained 
by minimizing L(A, z, x^^i), . . . , x^^^r', . . . , 

over z, x^^^\. . . , . . . , i?'^"^). Since strong du- 

ality holds in our problem we are guaranteed that the optimal 
value of MIN-COST-SW-NETWORK can be equivalently found 



by maximizing g{X) subject to A ^ [36]. Thus, if g{X) can 
be determined in an efficient manner for a given A then we 
can hope to solve MIN-COSTSW-NETWORK efficiently. 
Consider the optimization problem for a given A ^ 0. 

minimize f^z + ^ A^(i?('^'=) - x'-^"^) 

k=l 

s.t. < x|f ^ < Zy- < Cy, (^,J) GE\Tk€T 

{j|(»,i)eB*} {j|(j,')e-E-} 

R'^'^"^ G 7^5w, Tk G T. 

We realize on inspection that this minimization decomposes 
into a set of independent subproblems shown below. 

Nr 

minimize /^z — A^cc^T*-' 

k=l 

s.t. 0<xjf^<Zy<Cy, E*,Tk€T 

E -f^- E ^^''-<^r'\^^V*,TkeT 

U\{t.j)eE'} {jlUAeE-} 

(6) 

and for each Tk G T, 

minimize A^i?'-^''-' 

subject to R'-'^"'^ G 7^5w (7) 

The optimization problem in (|6]l is a linear program with 
variables z and 0;'^^'=) for fc = 1, . . . , A^i^ and a total of (2|r| + 
l)!^^*! + |T||F*| constraints that can be solved efficiently by 
using a regular LP solver It can also be solved by treating it 
as a minimum cost network flow problem with fixed rates for 
which many efficient techniques have been developed [14]. 

However each of the subproblems in dTji still has 2^^ — 1 
constraints and therefore the complexity of using an LP solver 
is still exponential in Ns- However using the supermodularity 
property of the conditional entropy function H{XB\Xg), it 
can be shown that TZsw is a contra-polymatroid with rank 
function H{Xb\Xb'=) [37]. In addition, the form of the 
objective function is also linear It follows that the solution 
to this problem can be found by a greedy allocation of the 
rates as shown in [33]. We proceed as follows. 

1) Find a permutation tt such that Xk^TT{i} > ^k,-K{2) > 

■ ■ ■ > Xk,,r{Ns)- 

2) Set 

=F(X{,(i)}|X{,(i)}.) and 

— iJ(-'^{7r(l),...,7r(i-l)}|-'^{7r(l),...,7r(i-l)}0 

for 2 < i < Ns- 

The previous algorithm presents us a technique for finding 
the value of g{X) efficiently. It remains to solve the maximiza- 
tion 

max g(A). 

For this purpose we use the fact that the dual function is 
concave (possibly non-differentiable) and can therefore be 
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maximized by using the projected subgradient algorithm [17]. 
The subgradient for can be found as — x^T'' -' [17]. 

Let A* represent the value of the dual variable A at the 
I*'' iteration and 6i be the step size at the i*'' iteration. A 
step by step algorithm to solve MIN-COST-SW-NETWORK is 
presented below. 

1) InitiaUze AO >z 0. 

2) For given A' solve 

Nr 

minimize f^z-J^i^kf^'^'^ 
fe=i 

i,j)eE\TkeT 



S. t. 0< ^ < Zy < Cy , 



(Tk) 



{j|(»j)6£*} {j|(j,*)6B*} 

for ieV*,TkeT 



using an LP solver and for each Tk G T, 



An) 



minimize (Xl)'^ R^'^"^ 



(9) 



subject to iJ^^*--) e 7^5w 

using the greedy algorithm presented in (H)). 
3) Set A^+i = [XI + 9,{R^^^^ ~ x^^'''>)]+ for all Tk £ T. 

Goto step 2 and repeat until convergence. 
While subgradient optimization provides a good approxi- 
mation on the optimal value of the primal problem, a primal 
optimal solution or even a feasible, near-optimal solution is 
usually not available. In our problem, we seek to jointly find 
the flows and the rate allocations that support the recovery of 
the sources at the terminals at minimum cost. Thus, finding the 
appropriate flows and rates specified by the primal-optimal or 
near primal-optimal z, x^^^'> , . . . , x'^'^" r\ R^^^\ . . . , i?^^"/?.) 
is important. Towards this end we use the method of Sherali 
and Choi [38]. 

We now briefly outline the primal recovery procedure of 
[38]. Let for j = 1, . . . , be a set of convex combination 
weights for each k > 1. This means that 

k 

^/i*' = 1, and yu^ > 0. 



We define 



Jjk = tJ-'^j/Ok, for 1 < j < fc, and fc > 1, 



and let 



Mk 



i{lok - 7(i-i)fc : i = 2, . . . ,/c}. 



Let the primal solution returned by subgradient optimization 
at iteration k be denoted by the vector {z,x,R)k and let the 
fc*'* primal iterate be defined as 

fc 

{z,x,R)k^^fi^{z,x,R)j for fc > 1. (10) 

Suppose that the sequence of weights yu*^ for fc > 1 and the 
sequence of step sizes Ok,k > 1 are chosen such that 

1) Ijk > 7{i-i)fc for all j = 2, . . . , fc for each fc. 

2) A7^^'^ ^ 0, as fc ^ cx), and 



3) 7ifc ^ as fc 
S>0. 



oo and jkk < S for all fc, for some 



Then an optimal solution to the primal problem can be 
obtained from any accumulation point of the sequence of 
primal iterates {{z,x,R)}. 

Some useful choices for the step sizes 9k and the convex 
combination weights that satisfy these conditions are given 
below (see [38]). 

1) 9k = a/{b + ck), for fc > 1 where a > 0,6 > 0, and 
c > and fi'^ = 1/fc for all = 1, . . . , fc. 

2) 9k = fc"", for fc > 1 where < a < 1 and /xj' = 1/fc 
for all 7 = 1, . . . , fc. 

The strategy for obtaining a near-optimal primal solution 
for the MIN-COST-SW-NETWORK pvoblem is now quite clear. 
We run the subgradient algorithm in the manner outlined 
above and keep computing the sequence of primal iterates 
{(z,x, R)}fc>i and stop when the primal iterates have con- 
verged. 

B. Results 

In this section we present results on the performance of 
our proposed algorithm. We generated graphs at random by 
choosing the position of the nodes uniformly at randomly from 
the unit square. Two nodes were connected with an edge of 
capacity 40.0 if they were within a distance of %| of each 
other and were connected with an edge of capacity 20.0 if they 
were within a distance of 0.3 of each other The orientation 
of the edges is from left to right. A certain number of nodes 
were declared to be sources, a certain number to be terminals 
and the remaining nodes were used for relaying. 

Let the random vector at the sources be denoted by Z = 
(Zi, Z2, ■ ■ ■ , Zns)- As in [39], a jointly Gaussian model was 
assumed for the data sensed at the sources. Thus the pdf of 
the observations is assumed to be 



f{zi,Z2,---,ZNs) 



1 



2^'^Vdet(Czz) 
X exp ( - - (z - iifCzl{2 - m)), 

where Czz is the covariance of the observations. We assumed 
a correlation model where Czz{iii) = cr^ and Czzihj) = 
(7^ exp(— cd^ ) when i ^ j (where c and [3 are positive 
constants and dij is the distance between nodes i and j). It is 
further assumed that the samples are quantized independently 
at all source nodes with the same quantization step A that 
is sufficiently small. Under these conditions, the quantized 
random vector X = {Xi , X2 , ■ ■ ■ , ) is such that 

H{X)^h{Z)-Ns\ogA 

as shown in [40] where H{X) represents the entropy of X and 
h{Z) = ^ log(27re)^''' dot (Czz) represents the differential 
entropy of Z. We can also express the conditional entropy 

HiX,\X,.) « ilog f(2.e)^^-|B-| det(C^z) 
2 V aet(GzBcZBc) 
-{Ns -\B'\)logA 
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(a) 



1000 

Number of iterations 



(b) 

Fig. 1 . (a) Network with ten source nodes (in red) and three terminal nodes (in 
green). The light edges have capacity 20 and the dark edges have capacity 40. 
All edges are directed from left to right and have unit cost, (b) Convergence 
of the subgradient algorithm to the optimal cost. 



We used these conditional entropies for the Slepian-Wolf 
region of the sources. 

Figure |l(a)| shows a network consisting of 50 nodes, with 
10 source nodes, 37 relay nodes and 3 terminals. We chose 
(T^ ~ 1, c — 1 and l3 ~ 1 for this example. The quantization 
step size was chosen to be A = 0.01. The cost of all the edges 
in the graph was set to 1.0. 

For the subgradient algorithm, we chose /i*^ = 1/k for all 
j,k and the step size 6k = The averaging process 

ignored the first 50 primal solution due to their poor quality. 
We observe a gradual convergence of the cost of our solution 
to the optimal in Fig. |l(b)| 

1) Remark 1: If one uses a regular LP solver to solve 
the MIN - COST - SW - NETWORK problem, as 
noted above, the complexity would scale with the number 
of variables and constraints, that grow exponentially with the 
number of sources. However, one is guaranteed that the LP 
solver will terminate in a finite number of steps eventually. 
Our proposed algorithm uses the subgradient method with 
step sizes such that the recovered solution will converge to 
the optimal as the number of iterations go to infinity [17]. 



In general, it does not seem possible to claim convergence 
in a finite number of steps for this method. A discussion 
around convergence issues of the subgradient method can 
be found in Chap. 6 of [17]. We point out that in practice, 
we found the algorithms to converge well. Note also that 
even the description of the LP requires space that increases 
very quickly, therefore using the LP formulation becomes 
impractical even with a moderate number of sources. 

C. Discussion about Fujishige 's algorithm 

We now discuss the work of Fujishige [32]. Towards this 
end we need to define the following quantities. A polymatroid 
P is defined as a pair {A, p) where A is a finite set and p is 
a function from 2"* to the positive reals, M-|_, that satisfies the 
axioms of a rank function. A vector a e , with entries 
indexed by the elements of A is called an independent vector 
of P[A, p) if Y^eeB "(e) ^ PiB), for all B <Z A. 

Suppose that we have a directed graph G = (V, E, C), with 
linear costs fij,y{i, j) G E, (as defined above) with two vertex 
subsets Vi (source nodes) and V2 (terminal nodes). Suppose 
that for each Vi,i — 1,2, a polymatroid Pi{Vi,pi) is defined. 
An independent flow is a triple {a, S, (3), such that: (i) a is an 
independent vector of Pi and is the vector of flows entering 
the network at the source nodes, (ii) (3 is an independent vector 
of P2 and is the vector of flows absorbed at the terminal nodes, 
and (iii) (5 is a flow vector such that flow balance is satisfied at 
each node in G. The algorithm in [32], returns an independent 
flow of maximum value, whose cost 5 is minimum. 

In the case of a single terminal, this algorithm can be used 
to solve the MIN - COST - SW - NETWORK problem 
(as noted in [31]) as follows. The set of source nodes and the 
conditional entropy function, specify a contra-polymatroid. An 
equivalent polymatroidal representation can be found without 
much difficulty (see [31]). Thus, these specify Vi and Pi. 
If there is only one terminal, then one can simply define a 
(trivial) polymatroid on it. This specifies V2 and P2- 

The situation is different when one considers network 
coding and multiple terminals. The algorithm in [32], is only 
guaranteed to return one set of flows that satisfies flow balance 
at all source nodes, internal nodes and the terminals. It is possi- 
ble to show the existence of instances where one set of flows 
will not simultaneously satisfy all the terminals, when one 
considers the MIN -COST- SW -NETWORK problem. 
An example can be found in Figure 6 in [13]. Moreover, the 
objective function in MIN - COST - SW - NETWORK, 
penalizes the maximum of the flows across the terminals at 
each node, which is different from the one in [32]. Thus, it is 
unclear whether this algorithm can be adapted to our problem 
in a straightforward manner 

IV. Quadratic Gaussian CEO over a network 

In general, the problem of transmitting compressible sources 
over a network need not have the requirement of lossless 
reconstruction of the sources. This maybe due to multiple 
reasons. The terminal may be satisfied with a low resolution 
reconstruction of the sources to save on network resources 
or lossless reconstruction may be impossible because of the 
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nature of sources. If a source is continuous then perfect 
reconstruction would theoretically require an infinite number 
of bits. Thus the problem of lossy reconstruction has also been 
an active area of research. In this section we shall consider the 
quadratic Gaussian CEO problem [16] over a network. We start 
by outlining the original problem considered by [16]. We then 
present the minimum cost formulation in the network context 
and present efficient solutions to it. 

Consider a data sequence {X{t)}'^i that cannot be ob- 
served directly. Instead, independent corrupted versions of the 
data sequence are available at a set of L agents who are in 
communication with the Chief Estimation Officer (CEO) over 
different communication channels. The agents are not allowed 
to cooperate in any fashion. Suppose that the CEO requires 
the reconstruction of {X{t)}'^i at an average distortion level 
of at most D. Here, the distortion level is a metric of the 
fidelity of the reconstruction. Suppose agent i communicates 
with the CEO at rate Ri. The CEO problem [41] is one of 
studying the region of feasible rate vectors , i?2 , ■ • • Rl ) 
that allow the reconstruction of the data sequence under the 
prescribed distortion constraint. As in the Slepian-Wolf case 
there is a direct link between the agents and the terminal (or the 
CEO). The quadratic Gaussian CEO problem is the particular 
instance of the CEO problem when the data source {X{t)}^i 
is Gaussian and the distortion metric is mean squared error A 
formal description of the problem follows. 

Let {X{t)}^i represent a sequence of i.i.d. Gaussian 
random variables and {Y.it)}'^^ = {X(t) + N,(t)}^^,i = 
1, . . . , Ns where {Ni{t)}'^i are i.i.d. Gaussian independent 
of {X{t)}^^ with E{N,{t)) = 0,Var{N,{t)) = af. Fur- 
thermore {Ni{t)}^-^ and {Nj{t)}'i^^ are independent when 

Let e > be a small real number. The i*^ agent encodes a 
block of length n from his observations (here, yi{t) 

denotes a particular realization of the random variable Yi{t)) 
using an encoding function : M" ^ {1, 2, . . . , [2"(^-+0j } 
of rate Ri + e. The codewords from the Ns sources are 
sent to the CEO who seeks to recover an estimate of the 
source message over n time instants {x{l),x{2), . . . ,x{n)) 
using a decoding function p„ : {1, 2, . . . , [2"(^i+'^)J} x • • • x 
{l,2,...,[2"(-""s+^)j}^M". 

Definition 2: A rate vector [Ri, . . . ,Rms) is said to be 
achievable for a distortion level D if for e > 0, there exists 
no such that for all n > no, there exist encoding functions 
: K" {1,2,...,[2"(^'+<^)J} and a decoding function 
gn : {1, 2, . . . , [2"(-«i+^)J } x • • • x {1, 2, . . . , [2"(^"s+^)j } ^ 
R" such that ^EJ2''t=iiX(t) - X{t)f < D + e where 

A complete characterization of the feasible rate region for a 
given distortion level D, denoted by 71(1)) has been obtained 
in [42] [43] and is given below. 



TZiV)^ U 7^p(rl,...,rJvJ (11) 

(ri,...,r„3)ejc-(x)) 



where 

TZTy{n,...,rNs) = !^{Ru...,Rns)-AC{1,...,Ns}, 

keA keA \'^~^l^ieA'= ^ / 

r 1^1- e"^''' 1 1 
T{V) = (ri, . . . , rwj : > 0, tf+E ^2 >^\- 

It is important to note that Tl{T>) is convex [43]. Thus, 
in principle the minimization of a convex function of 
R2, ■ ■ ■ , Rns) can be performed efficiently. 

We are interested in the quadratic Gaussian CEO problem 
over a network. In line with our general setup presented in 
Section |Il] the i*^ source node in S observes the process 
= {X{t)+N,{t)}'^j^ and encodes the observations 
at a rate Ri. Once again we are interested in the minimum 
cost network flow problem with rates such that they permit 
the recovery of the source X at the terminals with the desired 
level of fidelity, which in this case shall be measured by mean 
squared error. 

We start by highlighting the differences between this prob- 
lem and the minimum cost Slepian-Wolf over a network. In 
the previous subsection we observed that the work of Ho et al. 
shows that random linear network coding over a subgraph such 
that Ct; n TZsw ^ (p, VT; G T allows the lossless recovery of 
the sources at the terminals in T and essentially the Slepian- 
Wolf theorem holds even in the network case with multiple 
terminals i.e. any rate vector that can be obtained by joint 
coding can be obtained by distributed coding even when there 
are multiple terminals. However, an analogous result in the 
case of the quadratic Gaussian CEO problem does not exist. 
Furthermore, the rate region for the quadratic Gaussian CEO 
problem over a general network is unknown. As a simple 
example, we may have a network where two source nodes are 
connected to a common intermediate node. The intermediate 
node can then combine the quantized observations from these 
source nodes to generate a new quantized observation such 
that a lower rate is possible. Thus the rate region given by 
the classical Gaussian CEO problem may not hold as the two 
codewords may be fused to produce a new codeword that 
enables lower rate transmission. 

The first issue can be handled by assuming that there is 
only one terminal, i.e., Nji = 1 and Cti H TZ{T>) ^ cf) so 
that routing will suffice to transmit a rate vector belonging to 
Cti n72,(P) to the terminal Ti. Thus in this problem we shall 
not consider network coding. For the second issue we assume 
that the network operates in a separate compression and 
information transfer mode. The set of source nodes quantize 
their observations as they would in the original quadratic 
Gaussian CEO problem. After this source coding step, the 
network ignores any data correlations and routes the data as 
though it were incompressible. In general this separation of 
the compression and the information transfer is suboptimal, 
however it is likely to be a simple way of operating the 
network. 
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It is more convenient to cast this optimization in terms of 
the original graph rather than the augmented graph. The MIN- 
COST-QUAD-CEO-NETWORK problem becomes 

minimize Xij 

subject to < Xij < Cij, £ E 



- J2 = 0, z e (5 U {Ti})= (12) 

Xtj - ^ Xji < - i^Ti (14) 

i? e Tiiv) 

Here (fT2] i enforces the flow balance at all nodes in V except 
those in S* U {Ti}, (fTlT l enforces the constraint that at least 
Ri units of flow is injected at each source node i G S and 
(O ensures that at least X) jgs received at the terminal 
Ti. For the MIN-COST-SW-NETWORK pmhlem the total rate 
to be transmitted to Ti could be fixed to H{Xi, . . . , Xpf) as 
shown in the Appendix. However for the problem presented 
above fixing the total rate is not possible because of the nature 
of the inequalities specifying TZ{T>). A feasible solution to the 
optimization presented above would yield a routing solution 
such that the delivery of a rate vector belonging to Tl{T>) 
is possible at terminal Ti. The proof is similar to the one 
presented in the proof of Lemma [1] However even though the 
optimization under consideration above is convex, the number 
of constraints specifying TZ{T)) is exponential in Ns that 
would make a regular convex program solver inefficient. 



A. Solving MIN-COST-QUAD-CEO-NETWORK via dual de- 
composition 

We assume that the MIN-COST-QUAD-CEO-NETWORK 
problem is strictly feasible so that strong duality holds [36]. 
The Lagrangian with respect to the set of flow balance 
constraints that contain terms dependent on Ri,i E S for a 
given A is given by 



i(A,x,i?) =f^x + ^A,(ii', - ^ 



{3\{^,j)eE} 



U\U-^)eE} 



■^eS {3\{Tuf)eE} 



{3\U,Ti)eE} 

It is easy to see that finding the dual function g{X) = 
miria; L{X, x, R) subject to the remaining constraints decom- 



poses as 

minimize f^x — Xi I Xij — x 

^{j|(Tij)e-E} {3\U.Ti)eE} 

subject to < x^j < Cij, V G E, 

{3\{i,3)(^E} {j|(j,Oe-E} 

and 



(15) 



XiRi + Ati X R, 



minimize 

ies ies 
subject to R£TZ(V). (16) 

The optimization in (fTsT i is a linear program that can be solved 
efficiently. To solve ( fT6] l we note that for a given (ri , . . . , rjvs ) 
it can be shown that the region TZv{ri, . . . , rjvs) (defined in 
(fTTl i) is a contra-polymatroid [37]. Therefore an optimization 
problem such as 

minimize w^R 

subject to i? e TZt) (ri , . . . , rjvs ) (17) 

can be solved in closed form by using the greedy alloca- 
tion algorithm presented earlier. Using this fact we show 
(see Appendix-II) that the optimization in ( fT6] l reduces to a 
convex optimization problem with Ns + 1 constraints that 
can be solved efficiently via Lagrange multiplier methods. 
It is important to note that the optimal value of the above 
optimization problem is — cx) if Wi < for any i. This 
is because the inequalities defining 7?.(P) do not impose 
any upper bounds on the individual rates Ri for i E S. 
Consequently the optimization in (fTST i has a finite optimal 
value only if A^ + Ati > 0,Vi G 5. 

It is clear based on the previous arguments that we can eval- 
uate g{X) efficiently. We now need to solve the optimization 

max g{X) 
subject to Xi >0,ieS, A^i > 0. 

For solving this maximization we use the projected sub- 
gradient method [17]. As noted in Section Hill the subgradient 
algorithm may not return a primal optimal or primal near- 
optimal solution. For primal recovery for the MIN-COST- 
QUAD-CEO-NETWORK problem we use the technique pro- 
posed by Larsson et al. [44] that generalizes the method of 
[38] to the case of general convex programs. We point out 
some differences between the two methods below. 

The method of Larsson et al. considers an optimization of 
the following form. 

min f{x) 
subject to hi{x) < 0,i £ T 
x£X 

where the functions / and hi,i £ T are convex and the set 
X is convex and compact. It assumes the Slater constraint 
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qualification condition, i.e., the existence of a strictly feasible 
point xi such that {xi E X \ hi{xi) < 0, i G X} and considers 
the dual function with respect to the constraints hi,i E T, 

9(u) = min f(x) + > UihAx) 



and then solves the maximization of the dual function 

max B(u) 
subject to u )^ 

by using the projected subgradient algorithm. Let x'^^^ denote 
the primal solution obtained at the A;*'' iteration. Section 3.2 
of [44] shows that for step sizes dk e ^1^] , A: > 0, < 

/^<M<oo,fc>0 the sequence of averages defined by 



k=0 



(18) 



converges to the primal optimal solution as n oo. The 
choice of step sizes is more limited in this method as com- 
pared to [38]. In our problem the hi functions are the linear 
inequality constraints in ( fT3b and ( fT4l i and the existence of a 
strictly feasible solution is assumed. However, the set Tl{T>) 
is convex but not compact. The condition of compactness is 
however rather technical and can be enforced by imposing a 
loose upper bound on the rates. In practice, while running 
the subgradient algorithm the dual variables A; , i S S and 
Ati were constrained to be larger than or equal to 10~^° at 
any iteration to ensure that the optimized rates were bounded. 
Averaging the solutions as in ( fTSl ) we observed a steady 
convergence of the algorithm to the optimal cost in our 
simulations. 



B. Results 

As in the previous section we generated the graphs ran- 
domly. However, there is only one terminal in the MIN-COST- 
QUAD-CEO-NETWORK problem since we want a solution 
based on routing. Figure [2(a)| shows an example of a network 
with 10 source nodes, 39 relay nodes and 1 terminal node. In 
this particular example we chose the variance of the source to 
be cr^ = 0.01, the sensing noise variance of the source nodes 
to be af = 0.005 for all i G S and the required distortion 
level D = 0.003. The capacity of the light edges is 11 and 
the capacity of the dark edges is 22. The cost of each edge 
was set to 1.0. For the subgradient algorithm the step size was 
chosen to be ak = YPk- The averaging process ignored the 
first 100 primal solutions. As demonstrated in Fig. |2(b)| there 
is a steady convergence of the subgradient algorithm to the 
optimal solution. 

V. Lifetime maximization for sensor networks 

WITH DATA DISTORTION CONSTRAINTS 

We now consider the problem of maximizing the lifetime 
of a sensor network when the terminal node needs to be able 
to reconstruct the data at a particular distortion level (related 
problems were studied in [26]). This problem is important 
in the context of sensor networks where nodes are typically 
battery-limited and are sensing correlated phenomena that 




(a) 




2000 3000 4000 5000 

Number of iterations 

(b) 

Fig. 2. (a) Network with ten source nodes (in red) and one terminal node (in 
green). The light edges have capacity 11 and the dark edges have capacity 22. 
All edges are directed from left to right and have unit cost, (b) Convergence 
of the subgradient algorithm to the optimal cost. 



need to be reconstructed at the terminal node. It has been 
studied in [45] when the rates of the sensors are fixed. As 
explained in the previous section, the sensor nodes observe 
independent corrupted versions of an i.i.d. Gaussian data se- 
quence {X{t)}^i and communicate at a particular rate to the 
terminal node. The operating rate vector, whose components 
consists of the operating rates of each sensor should be such 
that the terminal should be able to reconstruct 
subject to a mean squared error distortion constraint. We are 
interested in finding routes over which the data should be 
routed so that the reconstruction (with an acceptable level of 
fidelity) can be ensured at the terminal for the longest period 
of time before a node runs out of energy. 

There is an inherent trade-off between the choice of the 
operating rate for a given sensor and the energy consumption 
that occurs when the data from the sensor is transmitted to 
the terminal since the amount of energy consumption roughly 



10 



depends on the amount of distance or hops that the data has to 
travel.. This is best illustrated in Fig. [3] The sensor node 52 
that is closer to the phenomenon of interest has a better sensing 
SNR, but is far from the terminal. Therefore 5*2 requires less 
bits for quantization, however the data needs to travel a longer 
distance. On the other hand, sensor node 5*1 that is further 
away from the source but closer to the terminal has a lower 
sensing SNR and requires more bits for quantization, but its 
data needs to travel a smaller distance. Thus there is a clear 
tradeoff in how we would want to perform the rate and the 
flow allocation if we wanted to maximize the network lifetime. 



Source 




Terminal 



Fig. 3. The figure shows sensor nodes 51 and S2 at different distances from 
the phenomenon of interest. The red path corresponds to the data from 51 
and the blue path corresponds to the data from 52. 



In this section we formulate the problem of maximizing the 
lifetime of a network in the quadratic Gaussian CEO setting. 
As in the previous section we assume that the network operates 
by separate compression and network information transfer We 
are given the following. 



a) An i.i.d. Gaussian data source sequence and 



set of source nodes S that observe independent corrupted 
versions of {X{t)} given by {Y,(t)}^-^ = {X{t) + 
N,{t)} for i = l,...,iVs where {Ni{t)}'^^ are i.i.d. 
Gaussian independent of {X{t)}'^j^ with E{N,{t)) = 
0, Var{Ni{t)) = af. There is a single terminal node Ti 
that seeks to reconstruct the source such that the distortion 
under the mean squared error metric is at most D. The 
feasible rate region for this problem denoted 7?.(P) is 
given by ( fTTT i. 

b) The initial battery level, Ei of each node i G V and the 
following power consumption figures of interest. 



i) 
ii) 
iii) 



Ptxihj) - the power consumed when i transmits data 

to j at unit rate for all V, 

Prxih j) - the power consumed when j receives data 

from i at unit rate for all i,j G V and, 

Psense{i) " the powcr consumed when i uses an 

additional bit for quantizing its observations. 



We define the network lifetime to be the time until the first 
node runs out of energy. The resultant optimization problem 
that we call MAX-LIFETIME-DISTORTION-CONSTRAINT 



can be expressed as 
minimize F 

s.t. 0<a;.y<Cy, y{i,j)eE 

Xij — Xji = 0, i G V\S U Ti 

{j|('j)6-B} {j|0-,*)6£} 

^ ^ Xij ^ ^ Xji > Ri, i (z S 
{il(ij)6-E} {j|(i,»)eB} 

b"l('-j)6-E} b"IO",»)6B} jes 

X] Ptx{hj)Xij+ ^ Prx{j,i)Xji 

{j\{i,j)eE} {il(jV06B} 

+ PsenseRt < E,T , % S S 
^ Ptx{i,j)x^J 

b"l(«-j)e-E} 

+ ^ i e V\S 

R e n{v) 

where F denotes the reciprocal of the network lifetime. Note 
that [45] considers the lifetime maximization problem when 
the operating rate for each node j e is fixed. Therefore 
the problem becomes a linear program that can be solved 
efficiently. However we are interested in jointly optimizing the 
operating rates and the lifetime of the network. In the formu- 
lation above we note that the specification of the region TZ{T>) 
is non-linear (although convex) with exponentially (in the 
number of sources) many inequalities. In [26], the authors con- 
sidered a problem similar to MAX-LIFETIME-DISTORTION- 
CONSTRAINT and proposed suboptimal solutions for by ap- 
proximating the constraints of Tl{T)) by linear inequalities. 
The authors presented approximate linear programs that were 
obtained by strengthening and weakening these constraints 
and concluded that the true network lifetime was between the 
results obtained by solving these linear programs. However the 
number of constraints was still exponential in the number of 
sources that precluded solving large instances of the problem. 
In this section we present a solution to this problem based on 
dual decomposition. 

We point out that we are considering a strategy where the 
rate allocation is static. In practice, it may be beneficial to 
adapt the rate allocation over time to extend the lifetime of 
the network. 

A. Solving MAX-LIFETIME-DISTORTION-CONSTRAINT by 
dual decomposition 

We note that as in section |IV] we can dualize the appropriate 
flow balance and energy consumption constraints and compute 
the dual function for this problem efficiently by exploiting the 
contra-polymatroidal structure of TZ{V). However, in practice 
we observed that the simple projected subgradient algorithm 
for maximizing the dual function for this problem is far too 
slow to be practical. Therefore we pursue an alternate line 
of attack here. We actually minimize F^ instead of F and use 
the proximal bundle method [46] and perform primal recovery 
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as explained in [46]. The Lagrangian for a given Ai and A2 
becomes 

L(Ai, A2,x,i?) = 



r' + ^AiJi?, 

les ^ 

+ ^i^'i ( J2 

^ kes 



J2 

b1(i-j)es} 
{j\{Tu]}eE} 



2^ ^JT, 
{M3,Ti}&E} 



iGS 



2i 



E 

{j|(ij)es} 



and finding .g(Ai,A2) = minr.x.i? -^(Ai, A2, i?) subject to 
the remaining constraints decomposes as 

minimize 



where Z is n compact and convex set, tjjj is a convex function 
for j = . . . n. Let / be the dual function of this optimization 
problem with respect to the constraints (z) < 0, j = 1 . . . n. 
Note that the dual function is always concave and that we 
may not know / in its functional form. However we assume 
/ and a subgradient of / at any given point is available via 
an oracle. We are interested in solving the original convex 
optimization by finding max^g^ f{x) where A is a non-empty, 
closed convex set and performing primal recovery. Toward this 
end we use the following algorithm. 
Proximal Bundle Algorithm 

• Step 1: Let S > 0, m G (0,1). Choose an initial 
point set = a;°, and let fc = 0. Compute f{x^) 
and a subgradient sq at i". Define the 0*'* polyhedral 
approximation to / as fa{y) = /(a;°) + sjf (y — i"). 



Step 2: At the k 



th 



E^- 



-r^A2 

E ^ 

b1(ij)es} 



E - 



y'^^^ G argmax 



Iteration, compute 



fk{y) 



\\y- 



+ A 



ITi 



E ^^ij 

{j\(Ti,j)eE} 



E ^J"?^! 

{3\U-Ti)eE} 



^2^ 



E 

mi-j)eE} 

<c, 



+Ea 

subject to < Xij ^ ^ y , 

{j|(j,i)e£;} 
^(^J>y+ E 

{j|(j,«)6S} 



{jio\»)6£;} 



E 

{il(^i)es} 

E 

{j|(iJ)6-E} 



Xi 



V(i,j) 
= 0, 



i e U Ti 

Prx{i.i)xj^ < E,r, i e v\s 



where fj,k is a proximity weight. Store the Lagrange 
multipliers corresponding to this optimization denoted by 
Vj >0,l < j < k such that J2j '^j = 1- 
Step 3: Define 4 = .fk{y) - fix''). If Sk < 6 STOP. 
Step 4: Compute /(y*^^^) and a subgradient Sk+i at 
yfc+i ^jgo store the value of the primal variables cor- 
responding to y'^^^, denoted by z'^'^^. 
Step 5: If fiy''^^) — f{x'') > mSk, perform a 
SERIOUS STEP (SS) x''+^ = y''+^, else perform 
a NULL STEP (NS) x''+^ = x''. 
Step 6: Update the model 



A.+i(y) = min{/,(y), /(y'^+i) + ^^^(y - y'+^)}. 



(19) 



and 



minimize 7 (Aij + Aiti + Psense><2i)Ri 



ies 

subject to Ren{V). (20) 

The optimization problem in (T% can be solved by using a 
quadratic programming solver and the optimization problem 
in ( l20b can be solved as shown in Section |IVl We used the 
quadratic programming package offered by MOSEkQ for this 
part of the work. 

From the above decomposition we have an efficient method 
to evaluate ^(Ai, A2). It remains to evaluate 

maximize ^(Ai, A2) 
subject to Ai, A2 



. Step 7: Set k = k + 1 and goto Step 2 

The work of [46] shows that the aggregate primal solution 
obtained by computing '^j^'' produces an asymptotically 

optimal primal solution as fc ^ 00. For more details (such as 
the choice of the /ife sequence) and for techniques for reducing 
the storage requirements associated with this method, we refer 
the reader to [46]. 

We applied this method to our problem since we can 
efficiently evaluate the dual function and can compute a 
subgradient at each point as well. Note that in this method 
as well, there is the technical compactness condition on Z. In 
our problem since there are no upper bounds on the rates, our 
region is not compact. However as in the previous section we 
impose loose upper bounds on the rates by enforcing the dual 
variables to be larger than or equal to 10^^°. 



for which we used the method of [46]. We now briefly g Results 
overview the proximal bundle technique. 
Consider the convex optimization problem 

min ipoiz) 
subject to ijj{z) < 0,j = 1 . . .n 

ze z 



http://www.mosek.com 



We ran the previous algorithm on the same topology shown 
in Fig. |2(a)| We chose the variance of the source to be a\ = 
0.01, the sensing noise variance of the source nodes to be 
erf — 0.005 for all i e 5 and the required distortion level 
D = 0.003. The battery levels for all the nodes were chosen 
to be 200 and we set Ptxihj) — ^-0, Prxi^, j) — 0.5 and 
Psense = 0.001. As demonstrated in Fig. |4] there is a steady 
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tion of the paper. 




qI , , , , , , , 1 

50 100 150 200 250 300 350 400 
Number of iteralions 

Fig. 4. Convergence of the proximal bundle algorithm to the optimal value 

of r. 



convergence of the proximal bundle algorithm to the optimal 
solution. 

We note that the lifetime maximization problem can also be 
solved in a similar manner if one consider the case of lossless 
reconstruction with multiple terminals. 

VI. Conclusions and Future Work 

We considered the problem of jointly allocating rates and 
flows at minimum cost for distributed source coding problems 
over a network. In particular, we considered (a) the Slepian- 
Wolf problem, (b) the minimum cost quadratic Gaussian CEO 
problem and (c) the problem of maximizing network lifetime 
when a source needs to be reconstructed within a quadratic 
distortion constraint. These problems are of interest in domains 
such as sensor networks where the data that is sensed by 
different sensors is typically highly correlated. The feasible 
rate region of distributed source coding problems is typically 
specified by a number of inequalities that is exponential in the 
number of sources that makes these problems hard to solve. 
We presented an approach based on dual decomposition that 
uses the special structure of the rate regions to efficiently com- 
pute the dual function. Finally, we demonstrated approaches 
for maximizing the dual function using the subgradient algo- 
rithm and the proximal bundle algorithm. 

It would be interesting to investigate algorithms along the 
lines of those considered by Fujishige [32] (that do not use 
dual decomposition), for the problems considered in this paper 
and study whether they have lower complexity. In all the 
problems we considered, we were able to decouple the rate 
allocations from the flow allocations. This essentially happens 
because the two sets of variables have a limited interaction 
via simple linear inequalities. As pointed out by a reviewer 
there may be other problems that may lend themselves to this 
kind of decomposition, where the interaction between these 
variables is more complex. 
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VIII. Appendix-I 
Theorem 2: Consider a vector (i?i, i?2, ■ • ■ , Rn) such that 

^R^> H{Xs\Xso), for all 5 c {l,2,...,n}, and 

n 

Ri > H{Xi,X2, . . . , Xn)- 

i=l 

Then there exists another vector {Ri, R2, . . . , R„) such that 
Ri < Ri for alH = 1, 2, . . . n and 

^ i?- > H{Xs\Xs-), for all C {1, 2, . . . , n}, and 

n 

= H{Xi,X2, ■ ■ ■ ,Xn)- 

1=1 

Proof. We claim that there exists a Rj* e i?2, ■ • • , Rn\ 
such that all inequalities in which Rj* participates are loose. 
The proof of this claim follows. 

Suppose that the above claim is not true. Then for all Ri 
where i £ {1, 2, . . . , n}, there exists at least one subset Si C 
{1,2,..., n} such that, 

J2 Rk^HiXsASf). 

fees, 

i.e. each Ri participates in at least one inequality that is tight. 

Consider the subsets Si and ^2, i.e. the subsets for which 
the inequalities are tight for i?i and R2 respectively. We have 
by assumption, 

Rk + ^ Rk ~ ^ Rk + ^ Rk 

keSi keS2 fceSinS2 fceSiUS2 

= H{XsAXsO+H{Xs,\Xs^) 

< H{XsiUS2 \X(SiuS2y) + H{XsinS2 l^(SinS2)0 (21) 

where in the second step we have used the supermodularity 
property of conditional entropy. Now we are also given that 

Rk > H{X^SinS2)\X(SinS2)^)- (22) 

feGSinS2 

Therefore we can conclude that 

E < H{X^s,uS2)\Xis,uS2r)- (23) 
keSiuS2 

Now let 5i2 = 5i U 5*2. We have two cases 

a) 5i2 = {l,2,...,n}. 

In this case we have a contradiction since the conclusion 
above impHes X^Li ^ H{Xi,X2, X^). 

b) 5i2 C {l,2,...,n}. 

In this case consider applying a similar argument as 
before with S12 and S3, i.e. 

J2 R,+ J2r,^ ^ R,+ ^ R, 

keSi2 keSs S12US3 SianSa 

< H{X(^SiuS2)\X{SiuS2)'=) + H{Xs:,\Xsi) 

< H{X(^SiUS2US3)\XiSiUS2US3y) 

+ H{Xsj^uS2nS3\X{SiUS2nS3)'')- (24) 
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Now since 



keSi2nS3 



we obtain 



S1US2US3 



Rk < H{X(^s^^JS2US3)\^ {81032033)") ■ 



If Si U S2 U S3 = {l,2,...,n} we have the required 
contradiction otherwise we can we can argue recursively to 
arrive at the contradiction. Note that the process terminates 
since 5i U ^2 • • • U S*,! = {1, 2, . . . , n}. 

The above argument shows that there exists some j* such 
that all inequalities in which Rj* participates are loose. 
Therefore we can reduce Rj- to a new value R^,'^ until one of 
the inequalities in which it participates is tight. If the sum-rate 

red 



where A C {1,2, . . . ,n}, A cf). Therefore the minimizer of 
dTTl ) can be written as 

= /({^(l)}) 

= /({^(l), . . . , - /({7r(l), . . . , 7r(z - 1)}) 

for 2 < i < n 

where tt is a permutation such that W7r(i) > 'W7r(2) ^ • • • > 
WTr{n)- The optimization problem in dTTl ) becomes 



minimize w^t^^-^R^ 



4=1 



constraint is met with equality then we can set i? 



R 



otherwise we can recursively apply the theorem to arrive at 
a new vector that is component-wise smaller that the original 
vector (i?i, i?2, -Rn)- ■ 
We refer the S-W constraint Y.j Rj > H{Xi, X^s) 
as the sum rate constraint. From theorem |2] we realize that if 
there exists a rate vector (i?i, R2, . . . , i?„) S TZsw that does 
not meet the sum rate constraint with equality then we can al- 
ways find another vector (R^, R2, . . . , that is component- 
wise smaller and meets the sum-rate constraint with equal- 
ity. Now consider the constraint Q in the MIN-COST-SW- 
NETWORK problem. Instead of setting the flow-balance at 
s* to H{Xi, . . . , Xpfs) and at Tk to -H{Xi, X^s) we 
could have introduced the constraints 



1 ^ 1 - e"2r, I 

subject to ^ + > o > — , ri > 0, Vi (25) 

at ^ af D 

Let A^ denote the set {7r(l), . . . , 7r(i)}'^. The objective func- 
tion simplifies to 



E 



k£A' 



(26) 

We note that the first constraint in dZSl) has to be tight. To 
see this suppose ?'7r(i) > and that the constraint is not tight. 
Then we can reduce ?'7r(i) so that the objective function ( l26b 
is reduced. If ^'^(i) = 0, then the argument can be applied 
to 7-ir(2) by realizing that the term corresponding to i = 1 in 
the second summation above is zero and so on. Using this 
observation the optimization in (|25]| can be rewritten as 



>^Rj'',i^ s* and, minimize ^ ^'r(i)''7r(j) + log(l/-D) - u'^(„) log(l/cr|) 



{j|(»J)eB*} 



{3\U-^)eE-} 



E 



X 



b1(i,i)eB*} {J|(J,«)6-B-} 



E 



(Tu) 



< -Y^Rf.^^Tk 



and attempted to the solve the resulting linear program 
(without forcing the sum rate constraint to be satisfied with 
equality). Suppose that this optimization has a feasible point 
(z, X, R) where the rate allocation for some terminal does not 
satisfy the sum rate constraint. Then based on the previous 
observation we can conclude that we can replace i? by a new 
set of rate allocations R such that R > R and {z,x,R ) 
continues to be feasible with the same cost. In fact one can 
possibly find a new set of flows that may have a lower cost. 
To conclude this shows that it is sufficient to consider rate 
allocations that satisfy the sum rate constraint. 

IX. Appendix-II 

Consider the quadratic Gaussian CEO problem with n 
sources. For a given (ri, . . . , r„), it can be seen that the rank 
function of the contra-polymatroid specified by Tl{T)) is given 
by 

7€ ^ 2^k 



4=1 

2^(w;7r(4+l) - W^(4))log I — + 2^ 2 

4=0 ^.^J^n ^k 



1 1 " 1 „ e-2n 
subject to — — —2 ^ = 0, 



(27) 



Next, we form the Lagrangian of the optimization problem 
in ( |27] | above with respect to the equality constraint while 
treating the positivity constraint on the r, 's to be implicit and 
obtain the KKT conditions [36]. 

n 

Hr, i^) = E '^^{i)'^^{'>-) + ^'r(l) l0g(l/£') - W^(„) l0g(l/CT|.) 



4=1 



E("'^(»+i) " log ( — + E 



4=1 



1 - e 



X i^^j^j k 



H^k 

k£A 



^log- 



^fe=l 



1 



+E 



1 - e 



-2ri 



+ E 



l-e" 



keA" 



X 4=1 

Differentiating with respect to TttJ,) for i = l,...,n and 



14 



W7r(l) - I' — 



setting to zero, we obtain the following equations 
dL 

dL 



'x(i) 
2iy 



-2r. 



(k) 



fc-1 



7r(fe) i=l Z^{,r(l),...,7r(i)} 5^ 

Solving these equations, we obtain 



- log 5 

- log 5 

2 V»^^(fc)^;(fc) 



for fc > 2 



where x+ = max(.T,0). Note that the form of the equations 
is such that they can be solved recursively for a given value 
of V. Furthermore v is uniquely determined by the equality 
constraint. Therefore a simple grid search on v suffices to solve 
these equations quickly. We note that a derivation similar to 
the one above has been performed in a completely different 
context in [47]. 
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